An annotation scheme for discourse-level argumentation in research articles

In order to build robust automatic abstracting systems, there is a need for better training resources than are currently available. In this paper, we introduce an annotation scheme for scientific articles which can be used to build such a resource in a consistent way. The seven categories of the scheme are based on rhetorical moves of argumentation. Our experimental results show that the scheme is stable, reproducible and intuitive to use. 1 Introduction Current approaches to automatic summariza-tion cannot create coherent, flexible automatic summaries. Sentence selection techniques (e.g. Brandow et al., 1995; Kupiec et al. 1995) produce extracts which can be incoherent and which, because of the generality of the methodology, can give under-informative results; fact extraction techniques (e.g. Rau et al., 1989, Young and Hayes, 1985) are tailored to particular domains, but have not really scaled up from restricted texts and restricted domains to larger domains and unrestricted text. Sp~irck Jones (1998) argues that taking into account the structure of a text will help when summarizing the text. The problem with sentence selection is that it relies on extracting sentences out of context, but the meaning of extracted material tends to depend on where in the text the extracted sentence was found. However, sentence selection still has the distinct advantage of robustness. We think sentence selection could be improved substantially if the global rhetorical context of the extracted material was taken into account more. Marcu (1997) makes a similar point based on rhetorical relations as defined by Rhetorical Structure Theory (RST, (Mann and Thompson, 1987)). In contrast to this approach, we stress the importance of rhetorical moves which are global to the argumentation of the paper, as opposed to local RST-type moves. For example, sentences which describe weaknesses of previous approaches can provide a good characterization of the scientific articles in which they occur, since they are likely to also be a description of the problem that paper is intending to solve. Take a sentence like "Un]ortunately, this work does not solve problem X": if X is a shortcoming in someone else's work, this usually means that the current paper will try to solve X. Sentence extraction methods can locate sentences like these, e.g. using a cue phrase method (Paice, 1990). But a very similar-looking sentence can play a completely different argumentative role in a scientific text: when it occurs in the section "Future Work", it might refer to …